According to a known technology, a displacement amount is detected on a plurality of images obtained by shooting the same subject, and then the images are located based on the displacement amount and are combined, so that an image having a higher image quality than an image obtained by performing shooting one time is obtained.
As an image sensing device that can obtain a color image, there are a multi-plate type and a single-plate type. The image sensing device of the multi-plate type includes a color separation optical system such as a dichroic prism arranged behind a shooting optical system and a plurality of monochrome image sensing elements arranged on optical paths divided by the color separation optical system. The image sensing device of the single-plate type includes an image sensing element in which on-chip color filters of a plurality of colors, for example, RGB or YCM, that is, three colors, are aligned on a light reception surface. Since the image sensing device of the single-plate type may have only one image sensing element, and a dichroic prism or the like is not needed, the size and cost of the image sensing device are advantageously reduced.
Since an image that is read out from an image sensing element having on-chip color filters without being processed is called a color filter array image, and, from the color filter array image, it is impossible to obtain an image that can be viewed by a person, demosaicing processing is performed on the color filter array image, and thus an image called a demosaic image is generated. The demosaic image generates, for each of the pixels of the image, pixel values, for example, pixel values of R, G, B corresponding to the number of colors of the on-chip color filter. In other words, although in the color filter array image, only a certain single pixel value can be obtained from one pixel, in the demosaic image, for one pixel, the pixel values of a plurality of colors are included. In the example of R, G, B described above, the amount of information on the image is tripled by the demosaicing processing as compared with the amount of information on the image before the demosaicing processing.
JP2002-112007A discloses a method of performing demosaicing processing on a plurality of images and thereafter combining them. In the method disclosed in JP2002-112007A, the synthesis processing is performed by mainly utilizing the component of G color.
JP2006-157568A discloses a method of combining a plurality of color filter array images without performing demosaicing processing. Here, a displacement amount is detected between the color filter array images that are a target to be combined, and while the detected displacement is being compensated for, the images are added and are thereby combined.